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ABSTRACT 

AS part of Project IMPACT'S efforts to develop 
procedures for complying with the intact require«ents of PuDlic Law 
94-482, a case study vas nade of the Illinois .pccupatibnal curriculu* 
Project (lOCP). The top-down study traced the iOCP from its 
developers to its users and docunented measurable changes in the 1971 
versus the 1979 curriculun development behaviors of coaaunity college 
adainistrators in Illinois. The purpose of IOCP was to develop 
fork&ble process aodels that could be used as a guide for local 
Education agencies and the^ Estate education agency in developing and 
evaluating cnrriculua in occupational education. The philosophy of 
IOCP iras that educational adainistrators should follow a step-by-step 
procedure prior to aakiag a decisioni In order to aid adainistrators 
in carrying -ut this aodel process,; 12 products were created* The 
aajor products were five activity aahuals containing learning 
activities Vhich were introduced at workshops^ Evaluation of the IOCP 
showed that three 6f the four intended consequences of .using the IOCP 
aanuals were reallized: aore systeaatic planning, aore use of resource 
aaterials, and aore writing of specific aanageaent objectives. The 
intended conseqaence that was not realised was aore staff 
involveaenti Findings of Project IMPACT'S case study confiraed the 
project evaluation and elucidated factors that facilitated that 
iapacti-Ptobably the aost significant finding regarding the actual 
aeasureaent of iapact had to do with the iapbrtance of collecting 
pre- and post-data on the project in the saae fashion (open-ended 
questions) and the observation that even when significant differences 
in behavior of adainistrators were observed, it would be iapossible 
to ascribe the change solely to the influence of the IOCP. (KC) • 
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Abstract ^ 

In order to identify and develop procedures for complying with the 
impact requirements of Public Law 94-482, PROJECT IMPACT studied 
five problems: (1) how to define impact, (2) how to assess impact, (3) 
how to show cause and effect relatonships between research and devel- 
opment (R&D) activities and changes in the vocational education teach- 
ing-learning situation, (4) how to predict the probability of impact, and 
(5) how to facilitate the impact of R&D activities. The methods used in 
these studies were to review and discuss views of experts and litera- 
ture related to the problems and to analyze the process of impact in 
selected cases of R&D activities and exemplary programs in vocational 
education in Illinois. Findings of PROJECT IMPACT'S activities from 
August 1, 1978 to July 1, 1980 are reported in nine volumes: (1) Con- 
text and Principles of Assessing . Impact, (2) A Case . Study of the 
Illinois Occupational Curriculum . Project, (3) A Case Study of the 
Illinois Network of Exemplary Occupational Programs for Handicapped 
and Disadvantaged Students, (4) A Case Study of Illinois Projects in 
Horticulture, (5) A Case Study of Illinois Career Education Projects at 
the Awareness Level, (6) A Case Study of the Occupational Survival 
Skills Project, (7) Case Studies of Two Illinois School Districts with 
Innovative Vocational Education Programs (8) A Field Study of Predict- 
ing Impact of Research and Development Projects in Vocational a»'id 
Technical Education, and (9) an Executive Summary. 

Volume 2^ traces the Illinois Occupational Curriculum Project from 
its developers to its users and documents measurable changes in the 
j|974^ver-sus- l979.-currjcul.um .deyelopme^^^ behaviors of community college 
' administrators in Illinois. 
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Overview of PROJECT IMPACT 

In 1974, the U.S. Office of Education (USOE) asked the National 
Academy of Sciences (NAS) to assess the impact of approximately 250 
million dollars spent by USOE on vocational education research and 
development (R S D) activities during the ten years between. 1 965 and 
1974, The NAS Committee on Vocational Education Research and 
Development (COVERD, 1976) reported that the R S D of the decade 
studied had a positive impact on curriculum development but did not 
have documented, widespread impact on the knowledge, skills or em- 
ployability of large numbers of students. National evaluations of voca- 
tional R S D in addition, to the COVERD report have been similarly 
critical (Development Associates, 1 975; Rand Corporation, 1975; Comp- 
troller General of thej^ited States, 197^). Acting upon these reports. 
Congress passed the Educational Amendments of 1976 (Public Law 
94-482) which mandated that contracts for R D In vocational education, 
not be'^ allowable unless the applicant could "demonstrate a reasonable 
probability that the contract would result in Improved teaching tech- 
niques or curriculum materials that would be used in a substantial 
number of classrooms or other learning situations within five years after 
termination of such contracts" (Federal Register, 1977). 

PROJECT IMPACT is a state funded study designed to.develbp 
procedures for assessing the impact of vocational education research- 
and development efforts vocational edtication. The primary purpose 
of the study is to identify and develop procedures for complying wlth_ 
the impact requirement of Public Law 94-482. 

8 • . 



The. study addresses itself to- three problem- areas: (1) how to 
define impact, (2) how "to- assess impact, and (3) how to show a 
cause-and-effect relafion^ship between project activities and changes in 
the vocational education^^teachihg-fearning situations. - The study also 
addresses two subsidiary problems: (1) how to predict the probability 
of impact and (2) how to manage on-going contracts to increase impact 
probability. 

•The methods used to^ conduct ^ this study were to review literature 
related to the problem areas, to interview individuals who have ex- 
parience and expertise in tha problem areas, andTo analyze the process 
of impact of several programs of related projects which were funded by 
the Illinois Office of Education/Department of Adult, Vocational and 
Technical Education/ Research] and Development Section and one project 
funded by the Comprehensive Employment and Training Act (CETA). 

The major activity of this study was analyzing the programs 
(casss) of related R S D projects, foji the first year, it was decided 
to select two cases for "top-down" analysis -and two cases for "bot- 
tom-up" analysis (see Volume 1, Appendix A, p. 43). 'The project staff 
referred to. these types; of retrospective analysis as "tracking." • It was 
anticipated that two types of tracking would produce different, insightls 
about impact. For example, bottom-up trackiny might better Identify 
"bottlenecks"" to impact than would top-down t:racking while top-down 
tracking might be more effective in relating project intents and project 
outcomes. One staff member was assigned as "tracking manager"- for 
each of the four cases and w.as instructed' to keep a detailed log of 
her/his activities (e.g., identifying documents, retrieving! documents-, 
identifying key people and- interviewing them, recording dbta>, analyzing 
.data). 



^ominatioYis.'^for "the fifst four case studies were sought from mem- 
"^^Wrs^of PROJECT' IMPACT'S Advisory Jsommittee^ project consultants, 
and project staff members. The following cases, .were^ selected for 

— - • ^ 

"top-dowii" tracking: - ' . 

1. "A Research and DeveFopment Project in Occupational Educa- 
tion" (The Illinois Occupational Curriculum Project— 1.0. CP.) 
' * which was developed by Joliet Junior College and funded by 
them and by thd Illinois Department of Adult-,- Vocational and 
Technical Education in fiscal years 1970-72. - ^ 
■ 2. ' "The Illinois Network pf Exemplary Occupational Programs for 
Handicapped and • Disadvantaged Students." At the time the 
case study was initiated', the ''network was in its- fourth year 
of operation. It was in tjie ^dissemination" stage and 

K 

lOE/DAVTE had funded .Illinois .State University to coordinate 
dissemination for the. Network's ejght cJemonstration projects/ 

The third and fourth cases, which, were selected for "bottom-up" track- 

ing, were: 

I 

3/ .'Mllinois R^ojects in Horticulture." .lOE/DA'VTE had funded 
some twelve R S D projects in horticulture between 1967 and 
1978. Both private and public schools participated in these R 
& D efforts. 

4. "Illinois Career Education Projects at' the Awareness Level." 
I0E7DAVTE funded three or more -major projects in this area 
between 1970 and' 1978, and CETA funded one for the Illinois 
Department of Corrections in 1975. 

During its second year PROJECT IMPACT studied the "Occupa- 
tional Survival Skills Project" and "Two Illinois School Districts with 



Innovative Vocational Education Programs" and developed a model for an 
*''lmp\4ct Asse^ssment System for the Mlinois Board o'f Education/Depart- 
ment of Adult, Vocational and Technical Education/ Re^eanch^and Develop- 
ment Section ' ^ * < 

. PROJECT IMPACT'S activities for^ the 1.979 and 1980 fiscal "years 
are reported in nine volumes.' Volume I — Context and ^.Principles of 
Assessing .Impact-contains an Introduction to PROJECT IMPACT, a 
review of literature and views of experts ir regard to planned educa- 
tional change and impactT the methods used to conduct case studies, a 
concise listing of references used during Phase 1 activities, and ap- 
pended materials such as the initial project proposal and data collectipn 
instruments wl\ich were developed for use in case studies. Supple- 
merital ' reports of. the project are contained in the following volumes: 



- Volume 2 ~ 
Volume 3 — 

Volume 4 -7 
Volume 5 ~ 

Volume 6 ~ 

Volume 7 — 

Volume 8 ~ 

/ 

" Volume 9 — - 



A Case Study of the ''Illinois Occupational. Cur-* 
riculum Project" 

A Case Study^of the "Illinois Network of Exemplary 
Occupational Programs for H^dicapped and 'Dis- , 
advantaged Students" ^ . 

A Case Study of "Illinois Projects in Horticulture" . 

A Case Study of "Illinois Career Education Projects 
at the Awareness Level" 

A Case Study of the "Occupational Survival Skills 
Project" 

Case Studies of "Two Illinois School Districts Vyith 
Innovative ' Vocational Education Programs" 

A Field Study of "Predicting Impact of Research 
and Development Projects in Vocational and Tech- 
nical Education" 

Executive Summary of Volumes 1-8 and Conclusions 
and Recommendations for Assessing the Impact of 
Vocational Education Research and Development on 
Vocational Education 



A Case Study of the Illinois Occupational Curriculum Project 

The steps for tracking this case from the top down were to collect 
jiata^on the origins, development, field testing, evaluation, diffusion, 
adoption and future of lOCP. The study is divided into a narrative 
report, a summary of critical decisions, a chronology of major events, 

and an impact-data matrix of lOCP. 

» «. — 

1. Narrative Report of the Illinois Occupational 
Curriculum Project (lOCP) 

The narrative report on 10 CP is organized as follows: introduc- 
tion, general description, origins, development, formative evaluation, 
diffusion, adoption, future, summary and conclusions* 

1 .1 Introduction 

A Research and Development Project in Occupational Education, 
later known as the Illinois Occupational Curriculum Project (lOCP) was 
"actually a group of related projects which were funded by the Illinois 
Office of Education/Department of Adult, Vocational and Technical 
♦Education. - The Research and Development Section funded a "Phase-One 
Project for Planning" for ^- months in March of 1970, a "Phase-Two 
Project .for Materials Development" for 12 months in July of 1970, and a 
"Phase-Three .Pi^oject for Field Testing and Dissemination" for 12 months 
in July of 1971. lOCP was developed by Joseph A. Borgen and Dwight 
E.. Davis* At the time I OCR was developed, Borgen and Davix, were 
Vocational education administrators at Joliet • Junior College in Joliet, 
Ijlinois. Borgen, Davis and their R&D team produced numerous 
products. A listing of 13 of these products 'was contained in an lOCP 
bibliogragby (see Table 1). Many of these products were distributed 
• by the Illinois Office of Education/Department of Adult, Vocational- and 
Technical Education. However, the items designated 8,. ^, 10, 11 and- 
12 ctwhich were the major iOCP products) were published bV the 
McKnight Publishing Company in 1974 and distributed by them under 
-th^ title Planning,' Implementing and Evaluating Career Preparation Pro - 
- grams . Items 1, 4, 5, 6 and 7 are available through the Educational 
Resources Information Centerls (ERIC) Jj^glection of documents^ 
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Table 1 

c 

Bibliography of Illinois Occupational Curriculum 
' Project Reports and^ IPubl^^ 

"Proposal for Phase I of A Research and Development Project in Occupa- 
tional Education: The Development of Process Models for Decision-Making 
In Curriculum Development and Evaluation." lOCP, Joliet Junior College, 
Jol let, Illinois: February 1, 1970. (ED 050\270) 

"Proposal for Phase M of A "Research and Development Project in Occupa- 
tional Education: The Development of Proces^ Models for Decision-Mak- 
ing ip Curriculum Development and Evaluation." lOCP, Joliet Junior 
College, Joliet. Illinois: May 29, 1970. \ 

"Proposal for Pi:ase III of-A-Research and Development Project in Occupa 
tional Education: The Development of Systems Models for Decision-Mak- 
ing in Occupational' Curriculum Development and Evaluation.!' lOCP, 
Joliet Junior College, Joliet, Illinois:- April 23, 1971. . ^ , 
"Phase I Report of A Research and Development Project in Occupational 
Education: The Development of Process Models for Declsion-Makipg in 
Curriculum Development and Evaluation." Unpublished report by lOGP, 
Joliet Junior College, Joliet, Illinois: October, 1970. (ED 050 270) 
An I nvestigation of Curriculum Development and Evaluation Models with 
Implica tions Toward a Systems Approach to Cur riculum Development and 
Evaluation In Occupational Education ; Joliet, Illinois: Joliet Junior Col- 
lege, 197K . (ED- 060 201 J ~~ , . _ 

An Inv estigation of Decision-Making Practices in Illinois Junio r Colleges 
with I mplications Toward a Systems Approach to Curriculu m Development 
and Evaluation in Occupational Education : Joliet, Illinois: Joliet Junior 
College, 1d>1. (E^ 060 19$) '7~ - ~ ^ ^ . 

' An Inves tigation of Systems Designs and Management Techniques 
wi th Implications toward a Systems Approach to Curriculum develop- 
ment anci Evaluation in Occupational Education ; Joliet, Illinois:- Joliet 
Junior College, -1971.. (ED 060 200) ~~ • . . 

Manageme nt Strategies and Guidelines for Using lOCP Manuals ; Spring- 
fleld, Illln-^Ss: — State of Illinois, Board of Vocational Education and 
■Refiabflitatioii, 1?72. * - . . c / ^ 

Activity ' Manual for Occupatidha> Program Identification ; Springfield, 
Illinois:: State, of Illinois, J^oard ot Voc^iondl tducatibn and Rehabili- 
tation, 197^. . e . r. .-1 

Xctivfty- Manual for Occupational Program Development ; Springfield, 
Illinois' State of Illinois, Board of , Vocational Education and Rehabili- 

tationv 1972. \ ' : ' . r. • ij 

Activi ty^ Manual for Occupational Program Implementation ; Springfield, 
lllinojs: State of Illinois, Board of- Vocational Education and Rehabili- 
.tation, 1972. , . ' . . r. , « 

Activity .Manual for Occupational Program Evaluation ; 5P'"'"9T 
IlltnolsI.^State of Illinois; feoard of V6cational hqucation and Rehabili- 
tation, )^72... ^ . - , . ' . ^ .. ., 
"Final Report of the Illinois' Occupational Curriculum Project." Un- 
pub'lishiji report'by lOCP, Joliet, Jllfnois: JoHet Junior College, 1972. 
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The focus of lOCP was on the administration of career preparation, 
programs at the secondary and postsecondary levels. According to the 
"I OCP -Proposal for Phase One" (pp. T-3), the groups targeted for 
.imFjact from the R & D activities and products were local educatio'n" 
agency (LEA) and State educatton agency . (SEA) administrators in 
iliihois. However, no figures for those two populations were given in 
the lOCP proposal. lOCP received a" total of $227,297 in state (Illinois) 
ana local (Joliet Junior College) funds to carry out its R & D activities 
during 1970, 1971 and 1972. The State funds for those years came_ 
from the Research and Development Section. State funds for sub- 
sequent activities, came 'from another section. 

1.2 General Description* 

The stated purpose of lOCP was to develop workable process 
models that could be applicable as 3 guide, for, (1) local eduation 
agencies in program development and evaluation and for (2) the State 
education agency in planning and decision-making (p. 3 of the Proposal 
for P^se One). The stated objectives of lOCP were: 

0 To develop . process models for curripalum development in 
occupational eduation. 

o' To develop guidelines for the utilization and application of the 
process models. 

0 To conduct a series of evaluation workshops to assess the 
value of process models. _ ^. ^ 

0 To test the applicability^ of the process models in a pilot 

situation and , other .settings, 

' J 
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0' To develop a plan for dissemination and in-service training 
for curriculum planners in the utilization of process models. 

0 To promote research on related problems (p. 2 of the Pro- 
posal for Phase One). 

The philosophy of !OCP was that educational administrators should 
follow a step-by-step procedure prior to making a decision. This 
procedure should include determinations of resources, evaluative cri- 
teria, and constraints affecting each decision. The philosophy is known 
as "systems approach". The systems approach has evolved into, a 
theoretical mode! consisting of three phases—analysis, engineering and 
management. Other theories or models which were reviewed fof* lOCP's 

development were: 

0 Models for curriculum development 

the objectives- approach (Taba's model, 1962) 
the- product-development approach (?) 
„ the systems approach • (Department of Defense, 1961) 

0 Models for curriculum evaluation 
accreditation (?)^ 

— Tylerian- (Tyler, 1 942-1 9_) 

— management-sys'tems (Stufflebeam's modei, 1967) 
summative-composite (Stake's model, 1967) 

0 Models for social, and economic effects of a planned curriculum 

— UNESCO data reported by W. S. Bennet (no date) 

The products de;.'ignated as item's 4, 5, 6 and 7 on the lOCP 
bibliography in Table 1, were typed and reproduced on white paper and 
held together with plastic spines. Each document had a colored cover 
which contained a design as well as a title. 

The products designated as items 8, 9"", 10, 11 and .12-were re- 
leased in five (8i". x 11") paperback manuals. Each manual had a white 
cover with the title printed in a bright jontj <^ting color. The pages of 
each manual were in color-coded sections of white, blue and yellow^ 
oaper. The contents' of the manuals were presented in a branching 



PHASE 



1, 1970 - June 30, 1970 



PHASE 



July 1 . 1870 - June 30, 1971 



PHASE III 
July 1 , 1971 - June 30, 1972 



PROJECT PLANNING 



MATERIALS DEVELOPMENT 



FIELD TESTING 
DISSEMINATION 



State 
$24,550.00 



Local ^ 
is. 916. 00 



State 
$67,178.00 



Local 



State 



$16,950.00 $90,503.00 



Local 
$21,200.00 



Total = $31,466.00 



Total = $84,128.00 



Toial = $111 ,703.00 



Figure 1. The funding. pattern for lOCP 
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form of programmed instruction. Instructional activity statements were 
identified by a hierarchical numbering system and a page reference. 
The contents of the five lOCP activity "manuals are listed below. The 
manuals presented some seventy learning actlvitfes which covered these 
topics. 

0 Advisory Committees 

0 Ancillary Services , - 

0 Budgets 

0 Cataloging Materials and Equipment . , 
. 0 Competencies \ 
0 Cost InformLtion \ _ 

0 Courses/Programs ^' 
0 Credit by Examin^tFon 
0 Employment Trends 
0 Evaluation 
0 Factllties 

0 Instructional Equipment and Materials , 
. 0 Instructional Methodology \ 
0. Objectives ' ^ 

0 .Program Requests/Suggestions - 
0 Recruiting • . 
0 Scheduling 

0 Scheduling ^ . . , . 

0 Specifications 
0. Surveys 

0 Target Groups ^ 

When McKnighl' published the five lOCP activity manuals, they 
punched the pages- and placed alP five manuals in one hard-cover book 
in the form of 'a three-ring binder with a colorful cover. The lOCP 
products which were, released by the Illinois Office of Education were 
distributed free.- The, cost of the McKnight (1971) publication was 
.dollars.. 

The lOCP product.? designated as 6, 7 and 8 in Table 1 were 
prepared as parts of the Phase Two Report to .the Illinois' Board of 
Education. Copies pf the documents were also sent to I OCR con- 
sultants. Some of these consultants who were teacher educators pro- 



ceeded to use the products as assigned or recommended reading for 
college students. No procedures were ever developed for this use, 
however. • — 

The five activity manuals were considered the major product of 
lOCP, and extensive procedures were developed for their use. Pages^ 
l-in of the first lOCP -manual (item 8 in Table 1) contained detailed 
instructions on how to use the five activity manuals. In addition to 
these written instructions, the lOCP developers and the Illinois Depart- 
ment of Adult, Vocational and Technical Education decided to release 
the manuals through a series of workshops so that the potential users 
would receive further instructions from the verbal presentations of 
lOCP's developers. 

I 

' ^ .1.3 Origins 

The key personnel of lOCP were Joseph A. Borgen and Dwight E. 
Davis who served as codirectors of each of the related projects. Two 
additional staff members who guided the development of lOCP were 
Urban T. Oen who served as Resear<:h Coordinator and David A. 
Anderson who served as Project Coordinator. All four of these project 
staff members were experienced vocational educators and all held^gra- 
duate degrjees in education. > " 

The ideas" from which lOCP originated were conceived in the fall of 
1969 ,by Borgen and Davis when-they were- vocational education adminis- 
trators at the same Junior College in Illinois. iThese gentlemen had' 
been friends and colleagues for many years. In both thejr.,personal 
ahd professional discussions, they developed a concern about certain 
administrative practices." They became especially" concerned about 
practices related to the planrlirig and evaluation of career preparation 



By this time in history, the U.S. Congress (through the Voca- 
tional Amendments of 1968) and the Statej)f Illinois (through the 1968 
State Plan) had mandated changes for the improvement and expansion of 
vocational education. Consequently, many institutions began offering 
new or expanded career preparation programs at the secondary and , 
post-secondary levels, and many new institutions such as Area Vbca- " 
tional Schools and Community Colleges were begun. Both the old and 
new institutions were hiring new administrators for their career pre- 
paration programs. . 

The lOCP developers feit there was a very great need to improve 
and systematize the overall process of occupational curriculum develop- 
ment and wanted "to be involved in an investigation having definite 
value to the [yocational education] profession as a whole" (p. 8 of item 
1 in Tab^e T) . Although - there were no Requests for Proposals" (RFP's) 
in this area of vocational research.,-the lOCP originators proceeded to 
develop their ideas for a research, development, and demonstration 
project. They decided to consult with one of their mentors who was a- 
curric-ulum specialist and an Associate Professor in Vocational and 
Technical Education' at the University of Illinois. In the fall of 1969, 
These three persons met in the home of or.6 of the lOCP developers and 
"laid out a model for the project on the kitchen table." their philoso- 
- phy for curriculum, development and evaluation was detailed in a PERT 
_chart contained in. an iOCP planhing document dated October, 1970. 
(Dr. McGage, manager of the. Research and Development Section of the 
Illinois Department of Adults Vocational ^nd Technical Education, said^ 
that the completed^ PERT- chart for_all three phases of >lpCP "was so 
long that it went, clear around a house trailer" which was being used as 
a temporary office at Joliet Junior College.) . . 
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A tentative process model which summarized major curriculum 
planning activities was also developed. These activities included; 



0 Assessing the commuhity needs 

0 Asses.sing the feasibly ty of programs to meet needs 

^ Assessing the characteristics of program clients 
0 Assessing the program content 

0 Assessing the competencies required by instructional staff 



The diagram of the model and a discussion of it are contained in Ap- 
pendix D of the Phase-One Report of lOCP (See item 4 in Table 1). 

The lOCP staff conducted three major investigations before they 
developed the five lOCP manuals (See item 5, 6 and 7 in TableT)*. A 
rnost significant product of ihe investigation of decision-making prac- 



tices v/as the survey of 1*971 practices of program/curriculum developers 
and evaluators in Illinois Community Colleges. Randomly selected sam-^^ 
pjes of vocational educators, institutions and occupational programs 
resulted in a total of 48 respondents representing 8 occupational areas , 
and 27 colleges. The findings of the survey provided baseline data 
which validated the 'assumption that .there .was a great reed to improve 
and system'atize the 1971 practices.' For example, the survey found 
that the most common practice pf occupational curriculum developers was 
to rely on"" the "feelings" of (rather than facts presented by) their 
advisory committees (93.8% of the respondents). Ih^. second most com- 
mon practice was to copy pcograms in other institutions (70.8% of the 
respondents). Local ^manpower surveys were done by a^out half ^ 
(l'4^2%) of the respondents. Only a few responde^nts^collected any .data 
on stuc^nt characteristics during'curriculum development activities, and 
no one-fep^x^d the use of occupational or task analysis activities as 
being- one of their curriculum deveiop'ment procedures. The most com- 



mon evaluation practice which was reported again involved a heavy 
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reliance on advisory committees (79.2%). About half (52.1%) of the 
respondents. .reported asking students to evaluate occupational pro- - - 

grams. 

Based on document analysis and interviews with key personnel and 
consultants, it is clear that lOCP originated from a fe^lt heed at the 
local level and that these needs vyere verified by the basJine data 
collected. These data pinpointed where the problems in the 1971 prac- 
tices of curriculum development and evaluation existed, and the pro- 
posed solutions to improving these practices were based on well-es- ^ 
tablished curriculurr* theories and models. The details of how these 
solutions were developed follow. 

Ji.^ Development 

At the. time lOCP was developed, only one other similar product 
was known to exist. Vocational, Technical, and Continuing Educa - 
tion in Pennsylvania; A Systems Approach^ State-Local Pro- 
gram Planning was undertaken by Walter M. Arnold in 1969 but had not 
been released, as of a 1971 investigations fpr JOCP. 

Little technological hardware was required for developing lOCP. 
However, the Phase-Two report on, decis!on-mat<tng~practlces~(-sed-Uem_6 
in Table 1) acknowledged 'use of a computer for data analys*-.. Al-^ 
-though the lOCP documents are Visible products, they contain guide- 
lines ^for systematizing intellectual processes. The.develof^ment'of COCP 
is difficult'' ^o describe becadse the intelTectuui products of lOCP tran- 
'scend the physical' products. 

As the developers refined their process model -for development and 
evaluation of occupational curricula- ancT developed an overall plan for a 



project to demonstrate its use, they gained the support of their col- 
'lege's president. .In February of -1970,. Joliet Junior , College (JJC) 
.sought State R 6 D funds for the project. Interestingly, instead- of 
"waiting until the next fiscal year, the Research and Development- Sec- 
tion funded them with a four-month (March through June) jalanning 
grant of 21,550. An additional $6,916 was contributed by JJC for a 
combined total" operating .budged of 31,166. The project was imple- 
mented immediately but there was considerable difficulty finding ad- 
ditional staff who were willing to make a cojTimi.tment_ for only fou/ 
mdriths. Consequently, only $11,936 of the $31,166 planning grant was 
actually spent. Information as to what proportion of JJ_C's financial 
'resources, staff, and facilities w^re devoted to lOCP was not available.. 

Some facts about JJC are probably significant to the development 
of. ICOP JJC .is the oldf st junior college in the United States and it has 
always been innovative. JJC had participated in other R&D projects in 
the past and was involved in., others at the^ time I OC P' was^^ developed. 
The lOCP developers and^other staff members at JJC had close working 
relationships with major, res.eat-.ch instkutions (especialjy the University 
of Illinois)', and lOCP benefited from' the expertise of consultants, who 
wer^^affHtat^a-AviithJiiesejr^^ 
T-he-originaUde-vel6pment._plan .^ille4 for fpur^tra^ses^-iHajTm^^ 



"Model Development and Preliminary Evaluation";' /^Pilot Testing of the 
Mode!*', andr. "In-Depth Evaluation aqd Dissemination". Three phases 
^ (Planning, Development and pissemination) were funded by the^Re- 

search and Development Section. However, the' develo"pers conducted 

•— ' ;■ 

several evalyatjon-activiti^es a,t their own expense. 



During the Model Development Phase, the developers were able to 
pilot test the lOCP processes and products at JJC and revise them 
accordingly. After this was done, they field tested the fkve activity 
manuals with l^* school districts in Illinois (7 community college dis- 
trict^, 3 area -vocational centers, and '\ comprehensive hign schools). 
UDtP s~taff members mad on-site visitTTd gather data for the 

rufinement of 'the lOCP manuals. Regional evaluation conferences yvere 
held to bring together field test participants. The five manuals were 
rfevlsed before they were submitted to the Illinois Office of Education. 
Most of the revisions had to do with the directions for using the man- 
uals. Most of these directions were in the first of-^the five manuals. 
Manuals two through-five required few changes. 

' These five mar.:^als contained written procedures and sample forms 
to assist local vocational educators with 63 instructional management 
tasks. During the same period of development, the Illinois Office of 
Education asked the lOCP developers to "prepare a publication titled, ^ 
Writing MeasTjrable Objectives Tor Career Education , The publication, 
included many excerpts from »the lOCP manuals. Approximately 10.000 
copies were.dlsse!:ninated in Illinois. 

Throughout the planning aTicl development .phases' of lOCP, the 
developers provided, for both internal and external formative evaluation 
of their activities and products. The 'details of these evaluation efforts 
^are described below. 
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U5 Formative Evaluation of lOCP 
From the outset of lOCP, thie developers set target dates for the 
completion of tasks and identified check-points for various "go" or 
-"no-go" decisions. They also made budgetary provisions for external 
evaluation of all phases of lOCP, There v;ere numerous written reports 
of- a^^nternal nature which were not readily available for^ dorcument 
analysis in this case study.- There were also meetings where verbal 
reports were *given and critical decisions made. Some of these events 
will be related in the summary of critical decisions section of this 
study. It should be noted that there was never just one external 
evaluator v/ho served lOCP throughout all its phases. Instead, experts 
•fn tasks of all the various Components (sub-models) of the IpCP process 
model were identified and hired, as consultants. These experts then 
reviewed some, but never aU, of lOCP's R e D activities 



K6 Diffusion 

Some mention has already been made of che diffusion of lOCP 
prod^ucts other than the five activity manuals (i.e., the three inves- 
tigative reports of Phase Two arid the publication on measurable ob- 
jectives during Phase ThreeJ. This discussion will .be limited to. dif- 
fusion of the lOCP manuals. ' * 

During the 1972-73 academic school year/the IlKnois Department of 
Adult, Vocational and. Techriical Education sponsored 10 thr6e-day 
workshops on use of the lOGP manuals. An announcement of thef^e 
^workshops^ was mailed to vocational educators throughout the statP.. No 
(i^f ormat iog^^ to how many announcements were mailed was available. 
Somi 550 Pearsons attended these .-10 workshops in Illinois between 
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August 1972 and June 1973- During the next year, the Illinois Depart- 
ment of Adult, Vocational and Technical Education sponsored three more 
such workshops, Minnesota sponsored ,six, and Michigan and Wiscopsjn „ 
each sponsored one or mere. No information as to the total number of — 
lOCP workshop participants was available. It is documented that par- 
ticipants consisted of both state and local vocational education* adminis- 
trators and some teacher educators but that the only intended target 
group of the workshops was local vbcational education administrators 
(Davis, 197^*). Dgring the 1973-7^* academic year, the Illinois Depart- 
ment of Adult, Vocation r.i and Technical Education cpntractefl for 10 ad- 
ditional workshops for local vocational education leaders to be conducted 
by the lOCP develbpers but' not limited to the lOCP manuals. In this 
Instance, the same 25 local leaders, participated in all 10 workshops. 
It should be noted 'that only the field tests of the 1971-72 school • 

,1 

year were conducted from the $227,297 in funds from the Research and C 
Development Section for the original threerphased project. The work- 
shops which were sponsored in the 1972-73 and 1973-7^1 school years 
were funded with non-R 6 D money from Illinois or other sponsoring . / 
States' Departments of Adult,; Vocational and Technical Education. 

• During prior actiyities, the Illinois Department of Adujt^. Vocational 
and Technical Education had evaluated its dissemination function and 
decided that , it did^not Include publishing on ^ large scale. They had • 
established a policy of protecting author's copyrights arfd getting bids-, 
for the publishing of materials Intended for wide dissemination., "Tnis • 
procedure was foHowed with lOCP and in the spri.lg o'f 197^*, the five 
activity manuals were released to the McKnight Publisi)jng Company of 
BJoomington, Illinois. Prior to this tim?,,r;,^one could get a set of 



manuals unless they attended an IOC.P workshop". This strategy for 
dissemination was a conscious decision oh the part of Che Illinois Depart- 
ment 9f Adult, VocatTonal and Technical Education and the lOCP develo- 
pers. They thought it would facilitate. lOCP's impact if it belonged to an 
elite cadre. McKnigh\ wanted to follow "this 'same procedure on a na- 
tional scale,. They ajso v/anted to hire.the, lOCP developers to conduct 
these workshops, but the lOCP developers were Interested in pursuing 
other aspects of their careers." They did, however, agree to represent 
McKnight '^t ,the annual convention of the Amerjcan Vocational Associa- 
tion in Decenrber-of 1974 and to Introduce the publication, called 
Planning, Implementing and"~Evaluatihq Career Preparation Programs . 
TY\e b^Ok-was not ready by the time of the convention. This un- 
fortunate timing is thought to have impeded the impact of the pub' 

lication. • ' ' f 

.During the time between the last workshop and the availability of 
the McKnight .publicatVn, the Illinois Department of Adult, Vocational 

and Technical Education and the lOCP developers received numerous 

. ' . • , ^ ^ 

requests for manuals. Therefore^. they decided. to fill as many of these 

requests as possible by disseminating the stock of manuals which the 
State still had. In^afJ, 'approximately 3,500 sets of manuals were ais- 
seminated by the State- of- Illinois. Several sets were retained by the 
State for 'library loan copies from the "East Centerl Curriculum Manage- 
ment Center. One of the -former lOCP staff.members -reported that he 
had-^"plicat9d 100 sets so that^ he could "supplyr-.his cdleagues" and 
that he "still has a few of those sets." 'McKnight's marketing data on 
the numbeFoTf copies sold e^ach year were obtained from one. of the 
IO'CP authors and are as follows: 






Copies on 
Approval 


Regular 
Sales 




1974 


5 


fi77 
0/ / 


- 


1975 


40 






1976 


' 20 


1 /O 




1977 


4 


129 




1978 


4 


107 




1979 - 


10 


85 






71 


1,687 





1.7 Adoption and Summatrve Evaluation 

< 

The .final report on the results of the first ten lOCP workshops 
(see rtem 13 on Table' 1) was not available. The tracking of the 

adoption of lOCP processes and products was largely facilitated by~the 

, ' . — 1 

summative evaluations of lOCP which were conducteu by Davis in 1974 

■. . 

and 1976^. Interviews ° with State' staff, lOCP developers, lOCP con--. 
sultapts, and jOCP workshop participants were ^o helpful. - 

In .Davis' first study, he reported both pvert and covert objectives 
for participants and staff of the 1072-73. workshops as follows: _ 



/ A Study to Determine the Extent of Adoption of Activities from 
the Illinois Occupational Curriculum Project Manuals Among Selected 
Vocational Eduators in Illinois. Urbana: University of Illinois (un-. 
published graduate paper)-; 1974. 

- ^The^'^^Consequences of Adopting an Educational Innovation:- A 
Study of the Impact of the Illinois Occupational Curriculum Project 
Manuals. Urbana: University of Illinois (unpublished doctoral thesis), 
1976, 
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Overt Objectives for the lOCP Workshop Participants: 

K To describe payoffs that exist for loca! leaders who use a 
systematic approach to the identification., development, imple- 
mentation, and evaluation of occupational programs. 

2. To describe the development and organization of the lOCP 
materials* 

3. To v/rite measurable objectives using lOCP- materials and 
^tpchniques. 

^. To use the lOCP materials to identify job(s) competencies with 
the help of an advisory committee. 

5. To write a specific plan for the utilization of the lOCP 
material's during the coming year. * ' ' 

Covert Objectives for the 1972-73 lOCP Workshop Staff: 

1 . To convince (and ,by example, demonstrate to) vocational 
• educators the value of staff involvement in planning and 

executing various activities in the realm of instructional 
program development and evaluation. 

2. To convince (and by example, demonstrate to) ocupational 

^ educators the value of opening communication channels with 
the community. 

3. To convince (and by example, demonstrate to) occupational 
educators the value of setting specific targets and engaging 
in systematic planning techniques (Davis, 1974, p. 4). 

To what-extent all of these objectives were achieved is not known. 

Attempts to evaluate the success of the workshops were limited to an 

end-of-workshop^opinionnaire. The opinionnarie was designed to obtain 

participant .reaction to the workshop methodology and content. Overall, 

it was found that participant reaction to the workshops had been highly 

favorable. Prior to the 1974 study by Davis, no attempts had been 

made to learn the extent to which participants in the 1972-73 workshops 

had litilized the lOCP manuals and/or workshop training. In his 1974 

study, Davis investigated what utilization had actually been made of the 

lOCP manuals.' Davis conducted a mail survey to obtain information 

from participants about themselves and tiie educational institution with 
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which they were affiliated, about their evaluation of the'lOCP manuals, 
about their utilization of 19 of the 63 activities in the iDCP manuals, 
and about their willingness to share their adoptions and adaptations of 
I OCP activities. Respondents were encouraged to send exhibits of 
these adoptions and adaptations. This adoption investigation was 
limited by the fact that workshop rosters contained incomplete or il- 
legible mailing addresses for participants. Some of the missing address 
information was obtained from the Directory of Illinois Schools. Davis 
decide'd the* survey should exclude teacher educators and State staff" 
members who had , been workshop participants because "they. would' 
undoubtedly have had little opportunity to utilize^'the lOCP manuals in 
the manner in. which they were intended." The survey instrument was 
finally mailed to 265 individuals, and the number of survey resiponses . 
totaled 206 (77% of the sample). Findings regarding adoption or adap- 
tion are reported 'below. ' 

The sample .of 19 of the 63 I OCP activities which were used 1n the 
survey were selected because (according to Davis, 1974, p. 70) they 
were "the mos^ likely to be utilized." Every respondent indicated that 
he/she utilized one or more of the 19 lOCP activities, ^he six most 
utilized of these activities were: 

0 Organizing or Improving an advisory committee 
0 Writing measurable program objectives 
0 Writing student performance objectives 
0 Conducting a student follow-up survey 

0 Involving an advisory committee in program or course evaluation 

0 Developing a new program or course 

One or more of the 206 respondents indicated that they us'fed and 
were willing to share their work in 18 of the 19 lOCP activities, and 12 
actually sent examples of their work. 

1 ' ■ 30 
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In his 1974- study, Davis also found that workshop participants 




v. 

u - 


frequently shared their lOCP manuals wi^h colleagues who had not 
attended w\)rkshops. (Perhaps it should be again noted that the man-^^ 
uals were only distributed through v,/orkshops. ) The impact of lOCP 


* i 


i 


began to snowball as users of the lOCP manuals continued to share with 






others. According to the. lOCP developers, another strong force in the 


. , ] 




adoption of 10 CP processes and prodacts was tftat State staff 'J really 






pushed it as a tool for complying with requirements ibr 'the State's one 


-A 


f ' 


and five year plan." 






Davis (1976) decided to investigate the consequences of iOCP 






adoptions and adaptations by conducting a participant observation study 






by interviewing IOCP users and analyzing documents at four Illinois 




LEAs^an upstate community college, a downstate community college, an 






upstate area vocational center, and a midstate area vocational center. 
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Davis wanted to. see if the following intended consequences had oc- 




\ 


curred: 




'i^ - • 


0 More systematic planning for occupational course or program 
additions 

0 More staff involvement in program planning and evaluation 

activities ^ 
0 More 'use of resource materials related to the management of 

instructional programs 
0 More writing of specific management objectives 




f 

; / 
\ • 


In all, Davis conducted 59 face-to-face interviews and analyzed 
numerous documents at the four LEAs. He classified his findings as: 


— 


: 


0 " Desirable or undesirable 
0 Intended' or unintended 
0 • Direct or indirect 


i 




Davis found that most IOCP users considered the consequences of 


> 




adoption to be functional and therefore desirable, that most of the 


t 


f ■ 0 
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consequenctsts were intended, and that most of the consequences were 
indirect rather than direct, 'it was explained that positive student 
reactions to the improved organization of classroom instruction was an 
example of an indirect consequence resulting from the direct con- 
sequence of written statem^^nts^of instructional objectives. A direct 

■"consequence of an undesirable nature^hich was reported by an lOCP 
adopter was that his peers harassed hjm for doing extra' work. Ttie 
lOCP^-adopter said he thought that his peers were jealous of recognition 
he had received for his work. 

During a three-day visit at each of the LEAs in the 1976 study, 
Davis also observed that many staff members were using lOCP processes 
and products but were unaware of the source of the material. He also 
observed that many staff members used lOCP forms without following 
(or by ignoring) the accompanying procedural instructions from the 

HOCP manuals. 

Three of the four consequences of using the lOCP manuals which 
Borgen and Davis had intended were realized: more systematic plan- 
ning, more use of resource materials, and more writing of specific 
management objectives. The intended cons_equence which was not 
realized was "more staff involvement". In his 1976 study, Davis found 
' that lOCP workshop participants returned to their institutions and 
directed theiir~staff toTuse JOCP processes and products, and, co \- 
sequently he concluded that lOCP adoption activities had reinforced the 
authoritarianism of vocational educatjpn administrators. This finding 
then^ would be an example of a direct but undesirable arid unintended 
consequence. 

32 
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Findings of PROJECT IMPACT'S ~case study interviews confirmed 
the findings Davis made as well as shed light on a few others. It was 
reported that McKnight's publication^f ..the lOCP manuals, was being 
used as', a college textbook* in occupational curriculum and evaluation 
courses. It was also reported that sections of the lOCP maouals hjd 
stimulated at least two major publications in vocational edUcation-'-^one on 
evaluation and one on advisory ' committees. The examples would seem 
to constitute some desirable but unintended impacts on unintended 

groups. ' • ^ 

Most reports of lOCP users "being unaware of the source of the 
material v^ere not surprising. However, one of the lOCP developers 
observed a section of a handbook put out by a State office (not in 
Illinois) which presented lOCP material without. citing its source. While 
this is an example of positive impact of lOCP, it is questionable that 
the user was unaware of the source. This consequence might have 
been avoided. If that State, had had a policy which required a concise 
listing of resources which were utilized in the preparation of a 
State-funded document. 

According to some vocational educators in Illinois, the impact of 
lOCP in Illinois was not sustained and the potential impact of the 
McKnight publication of the lOCP manuals was never achieved because 
the method of dissemination changed. It was t' li opinion of several 
people interviewed that much of lOCP's impact was due to the con- 

fident, pleasant- and inspiring personalities of its developers. One 

* 

participant reported that watching and listening^ to them «fas "like being 

at a live broadcase of a Bob and Ray show." The lOCP developers went 

on to new experiences in other States and continued to model the lOCP 
# * 
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processes, .,^and did^ .undoubtedly, carry some of lOCP's impact with 
them. However, it is nearly impossible to asses^ the extent of this kind 
of impact or to rep icate it\ The fact that both of the lOCP develdplers 
went on to "top jobs" was surely due, at \ea^c in part,— to the notability 
they had attai.ned wth lOCP. _ ' ,^ 

-This, concludes the adoption report of lOCP. The impact data 
which were described in this section are summarized in the im- 
pact-data-matrix (section 4). The future-of lOCP is discussed 
5elbw. * ^ 7" 

K8 Future of lOCP 

According to the lOCP developers and the McKnight Publishing 
Company, Planning, Implementing and Evaluating Career Prepara- 
tion Programs was considered to be a "thin-market product." 
McKnight's decision to publish it had to do with serving the profession 
rather than making a profit: Sales of the publication have dwindled 
eVery^y^ar sWce it'came out in 1974 but there are plarts on the part of 
McKnight and the authors to revise it. As stated earlier in this re- 
port, the Impact of lOCP's processes transcend Us products. And it is 
these intellectual rather than physical characteristics of the innovation 
that make it difficult to predict or assess its impact in the ftiture. 

1.9 Summary, Conclusions and Recommendations 

for Further Impact !nvestigation of lOCP ^ — 

lOCP was actually three related R&D projects funded jointly by 
the Illinois Board of Education/Department of Adult, Vocational and Tech 
nical Education/Research and Development Section and Jollet Junior 
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College from March. 1970 through June. 1972. .lOCP related -workshops 
were also funded by the State in 1973 and 1974 but these were not R 6 
D activities. Documefft analysis and interviews with key persons associa- 
ted with !OCP'"provided a jrcat d^iTof informatioh.< which was relevant-' 
to assessing the impact lOCP had in Illinois and to what facilitated that 
impact. Probably the most significant finding regarding actual measure- 
ment of Impact had to do with the basline data on the 1971 practices of 
vocational education administrators who were planning, implementing and 
evaluating occupational curricula. To date, this survey has not been 
rep icated. If it were, it would be possible to test these pre and post 
lOCP measures for statistically significant differences. If significant 
differences in the desired direction were found, then it could ,be said- 
that practices affecting the classroom had undergone a measurable a"d 
desirable change. -Because of other variables^ however, it would- not oe 
possible to say that all of the observable change had been due to the 

impact of 10 CP. 

If a study to collect post data were undertaken, it would be cri- 
tical to use the exact methods which iOCP used to collect pre data, 
e.g., open ended questions should be used, rather than questions 
whiere a group of possible answers is supplied for each question (see 
the summary of Critical Decisions in section 2 of this report). 
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\ ■ " 2. Critical Hecisions Affecting lOCP 

« 

.. ' the 'decision . of "the Illinois .^Board of Education/Department of" 
Adult, Vocational and Technical Education 'Research and Development 
Section to accept proposals for R 6 D activities which had not come 
through the traditional RFP channels, was certainly a factor In lOCP. 
As one DAVTE staff member said, Borgen arid Davis were able just to 
"come In off the street" and present their idea. The decision to fund 
'lOCP three-q uarte rs of the way through a fiscal year (March, of 1970), 
while the jesearchers were enthusiastic and prepared to begin im- 
mediately, may also have been significant. 

An lOCP planning document (dated October 1970) listed ten recom- 
mendations for further project development. One of these recom- 
mendations was that the services of an outside evaluator be obtained to 
evaluate the project and that these services should be budgeted as 
consultant fees for Phase Two. While -an outside evaluator was never 
hired to monitor all three phases of lOCP, there were numerous times 
when an outside evaluator was hired to consult on some activities of the 

three phases. \ ■ - — 

On NovemberNo, 1970, the results of a pilot study for An 
Investigation of Decision^makirig Practices were reviewed. Project staff 
and consultants found tha^ the lOCP instrument was not getting at the 
question of decision-making. Many respondents said they did each 
activity and thought It was Important. The Instrument did not identify 
who made decisions or the sequence in which decisions were made. A 
decision was made to develop a new Instrument which was to be com- 
pleted by interviewing a stratified random sample of 49 LEA admlnls- 
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tratbrs and teacKers. Tfie findings verified the need for .systematizing 
and improving curriculum practices. The findings also provided a basis 
for longitudinal evaluation (although this -has not yet been done). 

The lOCP developers had proposed a fourth phase of the lOCP 
project for the 1973-7a year. This fourth phase was to have included 
an In-depth evaluation of lOCP. However, lOE/DAVJE^was satisfied 
with the positive impact lOCP had had and decided not to fund the 
in-depth evaluation. ' • 

The .decisiQDS_X.egacd.iJ35 dissemination of the lOCP manuals through 
workshops and later through McKnight were probably significant—but to_ 
what extent and in which direction is not yet documented. 

The decisions of the lOCP developers to pursue other career 
activities is thought to have had a negative affect on the impa9t of the 
McKnight publication. However, the continued modeling of lOCP pro- 
cesses by the lOCP developers i? thought to have a continued positive 
affect in the states where the developers introduced lOCP (Illinois, 
Iowa, Michigan^, Minnebota a.nd Wisconsin), and elsewhere.^ 
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3. Major Events in !OCP . « 

Original idea for lOCP was conceived. 

Piiase One: The Illinois Department of Adult, 
Vocational and Technical Education/ Research and 
^Development Section supported a literature search 
In an attempt to dentify curriculum development and 
evaluation models that could form a basis for a 
packet of guidelines for LEA's to use when develop- 
ing occupational programs. Potential consultants 
were identified and a detailed project plan was 
prepared for Phases Two and Three. 

Phase Two: Reports of three research inves- 
tigations were completed: 

a) curriculum development and. evaluation 
models appearing in the literature 

b) management systems and modeling tech- 
niaues useful in the design of instruc- 
tional program planning guidelines 

c) instructional program planning and evalua- 
tion practices in Illinois community col- 
leges. 

The findings of these investigations provided the 
necessary bases for the first draft of the five lOCP 
manuals, as well' as actual measurable baseline data 
on 1971 practices in curriculum development and 
evaluation.. j 

Phase Three: Fourteen school districts (seven 
community college districts, three area vocational 
centers, and four comprehensive high schools) were 
selected to field. test the lOCP manuals. lOCP staff 
members made regular on-site visits to gaiher data 
for the refinement of the lOCP manuals. Regional 
evaluation conferences were held to bring together 
field test participants. The five manuals were 
revised and submitreiir to DAVTE. 

During the same period DAVTE's Professional and 
Curriculum Development' Section sponsored Joseph 
Borgen .,and Dwight Davis to prepare W riting 
Measurable Objectives for Career Education . T he 
publication Included excerpts from- the— IOC P 
manuals. Approximately 10,000 copie:; were dis- 
semlnatecl in-llllnois. 
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j.uly 1972 - Ten three-day.. lOCP vyorkshops were help through- 

June 1973 out Illinois. Approximately 5$0-vocational educators 
participated alotjg with some ,teache)"'.educators and 
DAVTE staff. ' - 

July 1973 - The DAVTE Professional and Curriculum Development 

June 1971 Section sponsored Joseph Borg6n and^ Dwight Davis to 
conduct three orientation and, training worl<shops 
for lea's on the lOCP manuals. 

The Minnesota Division ^of Vocational and Technical^ 
Education contracted with DAVTE^ Joseph Borgen 
"^nd^T^wight Davis tojconduct six orientation and 
training workshops for LEA personnel on the lOCP 
manuals. Michigan and Wisconsin sponsored :simllar 
v/orkshops. ' ' • ' ^ ^ 

In the 'spring,^DAVTE released the lOCP manuals to^ 
McKnight Publishing, Inc. of Bloomlngton, Illinois, - 
for commercial publication. Davis completed a 
foliow-up study of 265 participants from the 1972-73 
lOCP. workshop series. This study utilized a mail 
survey method. The survey was intended to iden- 
tify the adoption of lOCP activities. 

1975 - 1976 Davis completed a jDarticipant observation study at 

four LFA's in Illinois to investigate the con- 
sequences of lOCP adoptions. 

1978 - 1979 PROJECT IMPACT analyzed lOCPTor impact factors 

and decided to replicate lOCP's 1971 investigations 
of curriculum practices to see if they had under- 
gone any statistically significant changes. This 
study Is scheduled to be completed in 1 980. 

1979 - 1980 PROJECT IMPACT replicated lOCP's 1971 survey 

and found statistically significant changes (In the 
I desired direction) in the activities carrjed out and " 
^ the use of resources by occupational program and 

curriculum developers in Illinois community colleges 

(See Postscript to lOCP). 



4. Impact^DaW Matr^^ I()CP 
IMPACTS 



'Groups 
Impacted 



Directly: 

' Vocational 
Education 
V Adninistrators 
in Illinois ^ 



Indirectly: 
.Vocational 
Students in 

• Illinois high 

schools and 
Viunior colleges 



Special interest 
groups : 

Textbook 
PuBlishers 



Directly: 
Teacher Educators 

. in University 
Departments of 
Vocational Ed 



Indirectly: 
University 
Students in 
Curriculum 
Courses 



Special interest 
groups: 



Writers of 
Vocational Educa- 
tion Literature 



Intended Impacts 



Qualitative 



1.1, 1.2, 
1.3, 1.4 ' 



Unspecified 



Publication 
of lUCP 
Manuals 



Quantitative 



Unspecified 
% of 

unspecified 
size of 
^ population 



Actual Impacts 



Intended 



Dual i tati ve Quanti tati ve 



1.1 positive 

1.2 negative 
1.2 positive 
1 .4 positive 



Positive stu 
dent reactions 
to improved 
organization 
of classroom 

8 in<;tnictiQn 




none 



none 



13.1, 13.2, 
13.3, 13.4 * 



13. 



Unknown 



14 



1974-79 
Mcknight 
disseminated 
1,758, copies 



15 



none 



16 



none 



17 



18 



Unintended. 



Qualitative I Quantitative 



19. 



19 



20 



21 



Adopted Phase 
Two reports 
and lOCP 
manuals as 
22 textbooks 



Studied Phase 
Two reports 
and lOCP 
manuals 

23 



Stimulated 
^vpc ed pupli- 
-^catidns on 

evaluation, 

advisory 
24 conmittees 



25 



Additional 
workshops in 
Michigan, 
Minnesota and 
Wisconsin 



26 



27 



Unknown^ 



28 



Unknown 



29 



Influenced 
three 

— pub^l4eat46ns- 



30 



'41 



* See impact-Data Matnxjpecjfication Sheet on page 35.. 
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f ^-f *impact-Data Matrix Speelfi cation Sheet fbc 
1.0 Qualitative Intended- Impacts . 

1.1 More systematic practices for occupational curriculum develop- 
ment and evaluation 

1.2 More staff involvement, in program planning and evaluation 
activfties . ^ 

1.3 More use -of resource materials related to the management of 
instructional programs 

1.4 More writing of .specific program management and course 
objectives ' * - 

13.0 Quantitative Intended Impacts 

13.1 1971-72 As field testers, 14 Illinois- school districts 

' adopted lOCP processes and products. 

13.2 1972-73 Approximately 100%^ of 550 workshop par- 

ticipants reacted favorably to lOCP in 
Illinois. 

e 

13.3 1973-74 Three more lOCP workshops were held in 

Illinois, six in Minnesota, and one (or more) 
in at least two other states ' (Michigan and 
Wisconsin) . \^ 

- 13.4 1971-74 Illinois disseminated approximately 3,500 sets 
, . of lOCP manuals. 



19.0 Qualitative Unintended Impacts 

19.1 One lOCP adopter reported that his peers harassed him. 
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5. Postscript to lOCP 

a? 

In January 1980, PROJECT IMPACT decided to replicate lOCP's 
1971 survey o.f curriculum development practices in Illinois Community 
Colleges. Dr. Urban T. Oen who had conducted the 1971 survey as 
Research Coordinator of lOCP agreed to serve as a consultant to PRO- 
JECT IMPACT to collect the follow-up data. 

Procedures for the surveys were as follows. First the Illinois 
State Board of Higher Educaation was asked for a list of community 
college occupational programs which had been approved for implemen- 
tation during the past five years. Then stratified random samples were 
drawn. Because there were so many more new programs in the five 
years prior to 1979 than in the five years prior to 1971, the sample 
sizes were unequal — 49 in 1971 and 76, in 1979. A detailed description 
of sampling and survey procedures 'and a copy of the survey form are 
contained in Appendix A. • 

There were 48 respbn'dentF (9779Tln thFT971 survey and 75 
(98.&%) in the 1979 survey. Resp'ondents were asked about their, 
activities and the resources they used when identifying, developing and 
evaluating occupational curricula. In 1971, it was found -that when 
respondents were asked ]f they carried out an activity or used a par- 
ticular type of resource, they would say they did. However, dif- . 
fererices in responses were observed when respondents were asked 
open-ended questions as to how they identified a particular occupational 
program for curriculum development, what did they do next to develop 
the program and how did they evaluate the new program. These dif- 
ferences in responses pinpointed the areas to which lOCP needed to 
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direct its efforts. For example,, lOCP found a great need to increase 
practices of conducting manpower surveys, assessing student charac- 
teristlcs, completing job analyses, and conducting follow-up studies on 
graduates. .„ I OCP ^also directed its efforts toward increasing the types— 
of resources used by curriculum developers in occupational education 
(e.g. the use of community organizations and trade unions in addition 
to an advisory committee. These activities and resources for curriculum 
development in occupational education are specified in Tables 1 and 2. 

The 197r study reported .data for 18 activities and 16 resources 
used during thre^e phases of curriculum development - a program iden- 
tification phase (1), a program and course development phase (D), and 
-an-evaluation-phase-.{E).--JJie.5ample proportions {%) for 1971 and 1979 
are reported in Tables 1 and 2. Net changes for each phase^are also 
reported. A net change of 7% or more was statistically significant (at V 
the five per cent level). 

In reviewing net changes across all three phases, it is noted that 
sometimes an activity or use of a resource underwent change in only 
one phase (i.e. activities 1, 2, 5, 8, 14, 16 and 17 and resources 2, 7, 
9, 11, 13, and 14). Sometimes an activity or a resource had net gains 
in two phases or net losses in two phases. A net gain of 7% or more in 
at least one phase indicated a significant increase in behavior. A net 
loss of 7% or more, indicated a significant decrease in a behavior. A 
few behaviors had a significant net gain in one phase and a significant 
net loss in another phase. These findings may merely indicate a 
change in the^ph.ase during which a behavior occurred rather than the 
level at which it occurred. Since" tFTeselnstances^wj^^ few. 



TABLE 1 



BeRavioral Changes In Activities Carrled-CXjt By 
Curr I cu I in Developers In Illinois Qamainlty Colleges 



Activities 



% 1971 (N=it8) 



% 1979 (N=75) 



% Net Change 







1' 


rv2 

D- 


t-3 
E 


1 


D 


C 
C 


1 
1 


u 


u 


CQnplet6Ci local iranpowBr 




* 




65.3 






+11.1 






survey ' " . 














+ 5.7 




2. 


LooKed at oia manpower 








12*0 










data 
















< 


3. 


Detertnlhed target popula- 


10.4 


2.1 




9.3 






+ 1.1 


- 2.1 




tion's size 


St, 
















4. 


Determined target popula- 


2J 


2.1 




5.3 






+ 3.2 


- 2.1 




tion's characteristics 



















5. 


Conpleted job analyslis survey 








9.3 






Ck 1 




6. 


Looked atxprograms In other 


8.3 


70.8 




26..6 


2.6 




+1 8 .^3^ 


-5-44.2 




Institutions 


















7. 


Recruited staff 


10.4 


66.7- 




9.3 


41 .3 




-1.1 


-25, ,H 


8. 


RecrOTted students 




68.8 




• 


89.3 






+20.5 


9. • 


rld.nncu Taciiiiies 


12.5 


43.8 




45.3 


16.0 


•> 


+32.8 


-27.8 


10. 


UBXCmiiriGQ Vwldt fxfJl J-iiK" 


2.1 


2.1 




42.6 


10.6 




+40.5 


+ 8.5 




to buy 


















!!• 


Hi red staff 


4.2 


62.5 




2.6 


42.6 




- 1.6 


-19.9 




LOfTpietea toi low-tp 5ur*" \ 






31 .3 


2.6 




58.6 


+ 2.6' 






vey of graduatci 












4' 


+ 1.3 




13. 


Conpleted fol low-up sur- 






H • * 


1 3 




1.3 






vey of dropouts 


















14. 


Asked students to eval- 






52.1 






65.3 








Uove pruyr<«ii 


















15. ' 


Evaluated program 






70.8 


13.2 




88.0 


+13c3 




16. 


Evaluated staf-f 






10.4 






34.6 






17. 


Asked employers to eval- 






22.9 






13.3 








uate program 


















18. 


Detennlneri a budget 


8; 3 


8.3 




21.3 


2.6 




+13.0 


- 5.7 


^ 1 


= ^^Identification pliase 


















2d 


= Development phase 


















3e 


^ Evaluation phase 



















+27:3: 

- 2.9 1 

+13.2 

+17.2: 
+24.2-11 

- 9.6 



.A net chanje of 7% or more was general ly considered significant at the .05 level. How^- 
4er! ^ I^e iiable to catpute standard errors ^^enever the 1971 or 1979 % v«s less 
'J~\r~=~"thanr5-or--gFealer than 95. _ 
^ O Bipty spaces = 0%. 
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TABLE 2 
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, J 


i:!:^;/ Behavioral Changes In Use of Resources 

r , ,^ Cu rr icutum Deyelope>5s in Illinois Community Colleges 

■ ' ' * 




• 


; I 


if - Resources % 1971 (N=il8) % 1979 {N=75) 


% Net Change'* 








D 


E 


r 


D E 


* ^ s 


■ ''<^, 

|v / >Aoyisory commiiiee - oi«o 




Q 7 
7.0 


DD.O 


x1 ft 7 






h;., jnieresieu parenis 








-la 1 






-■.*U;<j»- rracui.iy . , - h.z 


9 1 1 ^ 






— 9 0 


- 9 1 


1 


, cp'^^^y^'ty organizaiions 


17 "5 


1 "5 


1 ^ 


a.1 7 ^ • 
+ 1 /-k 0 


X 1 Q J. 1 ^ 
+ .1^0 + 1.0 




( 5. Industrial relations 8 other 
y / .related committees 


4.0 




2.6 


+ 4.0 


+ 2.6 




l\ 6. Interested businessmen 39.6 


6.3 21.3' 


2.6 


2.6 


-18.3 


- 3.7 +2.6 




t 7. Union & management 
organizaxions 




2.6 





+ 


2.6 




-o. otate .consultants 


^ 9 1 ^ ^ 

0.0 Z*l 0.0 


9 ^ 
Z . 0 


9 A 
Z • 0 


■> 

0.0 


-.9 7 ft C 


■ ■ 1 


n 9. Interested students* 


9.3 






+ 9.3 






10. Curriculum guidelines 2.1 


12.5 ' 12.0 






+ 9.9 


-12.5 




r Manpower data 


2.6 






+ 2.6 




i 


? 12. Local money • 


8.3 1.3 






+ 1.3 


- 8.3 


^.;'13i7 State 6 federal money 


4.0 - 






+ 4.0 




5 


f it<»» Facilities available 


4.0 






+ 4.0 






r. 1 Identification phase 














:D Development phase 














^ E &yaluation phase 














!i,Z.«A^neixhange of 7% or more was generally considered significant at the .Ois level. How- 
j^Teyer, we were unable to compute standard errors whenever the 1971 or 1979 % was less 
/ than 5 or greater than 95. 


? 


r * Empty spaces =0%. 
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let us focus the discussion., on those behaviors in which there were 
significant changes; and let us review what activities are usually con- 
sidered to be most important in developing occupational curricula • 

According to most authorities in the field, no new programs should 
be approved unless there is a demonstrated need for them (e.g. man- 
power data which show that* there is a greater demand than there is a 
supply). After the need for new programs .has been established, the 
feasibility of starting these programs must be considered (i.e. what are 
the societal resources and constraints which will affect the success of 
such programs). The content of the programs should then be deter- 
mined by ^TR(f characteristics of_the occupatien—fKe. by completing 
fuVictional job analyses or using published job analyses which have been 
V^^ated by job incumbents). Most authorities also specify that the 
characteristics of the intended learners should be analyzed in order to 
determine appropriate criteria for student selection and . appropriate 
methods and materials for instruction. Few authorities specify the 
recruitment or hiring of instructional staff as a step in curriculum 
development. Likewise determinations of budgets, facilities and equip- 
ment are not usually considered steps in curriculum development. 
Recommended evaluation procedures for occupational programs have long 
emphasized the need tQ„ask, students and employed graduates to 
evaluate curcicula. - 

Now when we look at Table 1. we see significant increases in such 
Important curriculum development behaviors^ as using manpower data 
completing job analyses .conducting graduate follow-up studies and 
asking students to evaluate programs. While there^ were no significant 



^3 

changes in the behaviors Yor determining the characteristics of intended 
learners (activities 3 and 4), there was a significant increase in student 
recruitment (activity 8). Table 2 shows a significant increase in the use 
of interested students as resources (resource 9). These findings would 
seem to indicate significant increases in informal assessment of the 
intended learner' group. 

' The significant decreases in the proportions of respondents who 
included recruiting and hiring of staff (activities 7 and 11) may indicate 
that they noJonger perceived these administrative tasks as steps in 
developing new occupational curricula. It may also be that new cur- 
ricula were developed^ with existing staff. The^ significant decrease in 
obtaining employer evaluations (activity 17) appears to be a tradv^-off 
for the significant increase in follow-up surveys of graduates (activity 
12). ~ 

The changes in resource utilization behaviors are summarized in 
Table 2. All fourteen types of resources were mentioned in the 1979 
study whereas only seven \ypes were mentioned in 1971. There con- 
tinqes to be* a heavy reliance )n advisory committees. The clearly 
significant changes seem to be the trepds toward the use of interested 
students, community organizations ^nd away from parents, businessmen 
and consultants. The most striking finding is the limited use of re- 
sources by the vast majority of respondents in both surveys. • The 
State of Illinois and many professional and trade organizations provide 
free educational services such as consultants and curriculum guides. 
These 1971 and 1979 surveys suggest a need to investigate the utili- 
zation of_-uSUch ser/ices by developers of occupation^; curricula. 
/ 
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Wher:*. the lOCP manuals were prepared for the McKnight publication^ 
the choice of title - Planning, Implementating and Evaluating Career 
Preparation Programs - indicated a change in terminology for the three 
phases of curriculum developmsnt which were identified as identification 
(I) development (D) and evaluation (E). However; the behavioral 
changes which lOCP intended to bring about such as more systematic 
planning, more staff involvement, more use of resource matenals and- 
more writing of specific management objectives did not change. In his 
1978 investigation of the consequences of lOCP adoptions, Davis con- 
cluded that lOCP's impact was negative and unintended on one behavior 
(more staff involvement in decision-making) and positive and intended 
on t.he other three behavior^. The 1971 and 1979 survey data offer 
additional information on the first three intended behavioral changesT 
(outcomes). If we to compare behavioral levels without regard for the 
r-hases in which the- -behaviors occurred, we can see if there were 
overall increases or decreases in the desired behaviors. PROJECT 
IMPACT was not successful in retrieving 1971 data on decision-making 
behaviors (see items 17-29 and 32-35 on the survey form) and, there- 
fore, could n(0 make; comparisons between-baseline and follow-up data to 

see if "the measures Indicatnd any changes in the level of staff in- 

-r • — 

volvement. However, baseline and follow-up data for "more systematic 
planning" (activities carried out) and "more use of resources" were 
obtained and compared for' overall Increases or decreases. These find- 
ings are summarized in Tables 3 and 4. The findings were that in 
1979, respondents \.2re less likely to copyolher programs and rely on 
advisory committees, and more likely tQ_rely on manpower data, oc^- 
cupatlonal analyses and program evaluation data than did the 1971 
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respondents. These findings would seem to Indicate that more sys- 
tematic ' planning was being done by . respondents in 1979 than . In- 1 971 . 
The 1979 respondents ased more types of resource materials In 1979 
than in 1971. , The behavior of writing specific management objectives 
was not_assessed with the form used for the 1971 and 1979 surveys. 




TABLE 3 

Changes In Activities Carried Out By 
Curriculum Developers In Illinois Community Colleges 





Activities 


1971 


1972 


Ovej;;all 
Change 


1. 


Completed local manpower 
survey 


— 54.2 


65.3 


+11.1 

• 


dm • 

f 


Looked at old mahDOwer 
data 


• 

6.3 


12.0 


+5.7 


.3. 


Determined target population 
size 


12.5 


9.3 


-3.2 


4. 


Determined target population's 
characteristics 


1.2 


5.3 . 




5 


Completed job analysis survey 


0 


9.3 


+9.3 


6. 


Looked 'at porgrams In other In- 
• stltutions 


79.1 


29.2 


-49. 9 


7. 


Flecrulted staff . 


77.1 


50.6 


-26.5 


8. 


Recruited students 


89.3 


20.5 


+20.5 


9. 


Planned faplllties — - 


56.3 


61 .3 


+5.0 


10. 


Determined what equipment 
to buy 




53.2 


+19.0 


11. 


Hired staff 


66.7 


45.2 


-21.5 


12.' 


Completed follow-up survey 
of dropouts 


31.3 


61.2 


+29.9 


13. 


Completed follow-up survey 
' of ciropouts 


U.2 


2.6 


-1.6 


in. 


Asked students to evaluate 
program 


1 

52.1 


65.3 


+13.2 


15. 


Evaluated program 


' 70.8 


101.3 


+30.5 


16. 


Evaluated staff > 


10.4 


31.6 


+24.2 


17- 


—Asked" employers to evaluate 
program 

* 


22.9 


13.3 


-9.6 


:i8. 


Determined a budget 




23.9 


+7.3 



TABLE 4 

Changes In Use of R( 
Curriculum Developers-ln lUino 

•Resourees — • 

Advisory committee 
Interested parents ^ 
Faculty 

Community organizations 

Industrial relations 6 other 
related committees 

Interested businessmen 

Union & nanagement 
, . organizations 

State consultants 

Interested students 

Curriculum guidelines 

Manpower data 

Local money 

State 6 federal money 

Facilities available 



lurces By 

Community Colleges 



1971 


1972 


Overall 
Changes 


254.3 


167.9 


-86. U 


16.7 


2.6 


-14. 1 


6.3 


1 .3 


-5.0 


0 


19.9 


+19.9 


0 


6.6 


+6.6 


15.9 


26.5 


-19. U 


0 


2.6 


+2.6 


23.0 


18. U 


-U.6- / 


0 


9.3 


+9.3 


U.6 


12.0 


-2.6 


0 

<• 


■ ?.6 . 


+2.6 


8.3 . 


1 .3 


-7.0 


0 




U.O 


0 




+U.0 
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Summary and Coh'clustons 

j The intended (mpact of \0C? was to improve the curriculum develop 
1 ' \ • ' • ' ^ . 

ment practices of occupational educators (especiaHy locahlevel adminis- 

trators tn secondary and post^secondary schools in Illinois). That 

1 » \ 

thiere wars a need to improve these -practices was only an assumption 
until lOCP collected base\ine data in 1971./ The baseline data pinpointed 
the levels at which curriculum activities were being carried out and the 
rates at which curriculum= development resources were being utilized by 
developers of occupational - curricula in Illinois community colleges. 
Comparable, data were^ gathered five years after lOCP^s State funding 
was ^terminated and statistically signlffcant increases In ^desirable cur- - 
riculum development behaviors Were found. It should be noted that 
while lOCP was cpnijdered^to be a major treatment variable^ there was 
no controlled experiment where lOCP was an independent variable 
without any intervening variables* . It should also be noted that the 
1971 and .1979 surveys were Umited to colmmunlty colleges; and, there- 
fore, may no^ be general izable to occupational educators at the secon- 
dary level/. The 1,979 survey findings -were an attempt to assess the 
impact of a completed R&D project which had provlded-4t£"Own-base — 
line data. We ''can only Speculate about ^when lOCP's impafljTreached Its • 
peak. I.n terms of developing procedures for assessing trci^ 
versus sustained impacts, it would have be^n valuable lu> know the 
results of a 1975 survey as wetll. 

Based upon pur experiences in PROJECT IMPACT at the University 
of liiinois, it is pbssible tOo assess the probable impact of R & D . 

activities. Collection of baseline data to validate assumptions ^of a 

. .... .... ^ , ^ 
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search program should be a required part of the funded activities of a 
project unless these data -were adequately presented in the research 
proposal. Unless such baseline data are available for malting post-pro- 
gram comparisons, it is next to impossible to assess impact {measureable 

changes}-. Because- -of -this-- all- too frequent lack of baseline data, 

evaluators are often: pressed to report dissemination and utilization data 
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Appendix: LOCP Procedures and 
Forms Used to Survey 
Curriculum Development 
Practices in Illinois Community 
Colleges 



sty 
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S ELECTION OF THE SAMPLE * ^ 

1. The Illinois Community College Board* was contacted and asked to 
furnish a computer print-out of a!l new programs approved for the 
past five years. The list sent to us not only included all nev; 
'programs but those with only title changes^ or program modifica- 
tions. Not knovying new programs from those with changes, we 

left them all in the sarpple and provided a means for eliminating 
such programs during the interviev/ process. 

2. A 122 page computer fi>rint-out was received. In order to group 
the programs by title, it was necessary to sort 2,562 listings into 
appropriate and similar programs by institution. From this con- 
solidation, a list of approved programs was developed. 

J. From the . list a master matrix table by instruction and by program 
was developed. ; 

4, A total of 73 programs were approved for five or more institu^tions. 

5. To coincide with the original- survey, all programs were placed 
under .the five major headings used in the State of Illinois; that 
is, Applied Biological and Agriculture Occupations; Business, 
Marketing and Management Occupations;,, Health Occupations, In- 
dustrial Oriented Occupations; and Personal and Public Service 

. Occupations. Each program was then assigned a number and using 
a table of random numbers, the programs to be used in the study 
were selected. Applied Biological and Agricultural Occupations had 
the fewest number approved i.e., five programs. In order to 
keep an even»ratio of programs selected, five became the common 
denominaton.__rhe^progcams- sheeted jD.y_area„,we^^^ ,„ 

Applied Biological and Agrfoulture Occupations 

Agriculture Supply/ ' 

Business , Marketing and Management Occupations 

Fashion Merchandising ^ ^ 

Certified Professional Secretary 
Secretarial Science 
Stenography 

Health Occupations 

Dietetic Technology 
Practical Nursing 

Personal and Public Service Occupations 

Food Service Management 
Fire Fighter 
Law Enforcement 
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Industrial Oriented Occupations 



Auto iBody Repair 
Architectual Drafting ^ 
Construction Technology 
Buildingv:Con$truction Technology 
Electron res Technology 

In addition ^to the main samples, alternates were also chosen. 

the next step was to determine which programs would .constitute 
the study sample.' To accomplish this, the schools within each 
program area selected were assigned a number .and again using a 
table of "random numbers, fiVe schools were selected in- each pro- , 
gram area. In addition possible alternates were identified in case 
a particular institution had failed to establish that particular 
program. 

Select ion' of the Training Sample and Training of the 'Interviewers 

r ' \ 

Ken Andrew, a graduate assistant from the University of Illinois, 
came for two days of* training. , The project procedures were 
thoroughily explained. The training sample was composed of College 
of DuPage and Elgin Community College. The College bf. DuPage- ■ 
and Elgin Community College interviews were on-site to provide 
face-to-face exposure. Since it would be impossible to do all on 
site interviews, the final time was devoted to calling schools via 
telephone' and completing the interview in this fashjon. A critigue 
followed each telephone call. Some simulation was also used. 

Collection of the Data 

— — ^ ■ . . 

A list of all schools to "fae contacted and the programs in..each . 
school was compiled. The telephone number for each school was • 
placed on the list. Each occupational dean of each school in the 
sample was contacted by telephone to explain the purposes of the 
study and to schedule the interviews. 

It come than one person helped establish the program, both were 
interviewed. Originally we intended to interview all parties in- 
volved but since many people had moved it was impossible to 
always contact each concerned person. In most instances we 
interviewed deans, division or department chairman, and lead 
teachers. 

The procedure for the interview was as follows. The project was 
explained and asked to respond in terms of program identification, 
program development and program evaluation. Key guestions were 
asked to help the respondent. The standard introduction and key 
guestions, ana forms are attached. One the forms used to record 
the data, an "X" was placed opposite the item in the seguence 
given to the researcher. For instance, if a person indicated to 
the researcher that he first looked at old manpower data, an "X" 
was placed in Column 1, row 38 on the "Identification of 
Occupational Education Curriculum Decision Making" form. 
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STANDARD FORMAT AND KEY QUESTIONS USED 
■ DURING THE PERSONAL INTERVIEWS 

A research project currently in progress at the University of Illinois is 
directing its efforts toward the identification of administrative deci- 
sion-making related to d^evelopment and evaluation of occupational educa- 
tion' programs. Many personhel from Illinois junior and community 
colleges* are being asked cto make contributions to the 'project. 

The project is funded by the Illinois Office of Education/Department of 
Adult, Vocational- and Tchnical Education/Research Development Section. 

One objective of the project is to identify those people making |de- 
cisions, the different decisions made, and by whom. Another objective 
is- to identify? all those factors which are considered in making de- 
cisions, A third objective is to identify the philosophy, rationalel and 
organizatipnal' structure of the cievelopment-^(3j administration of /unior 
college occup'atioFtaf-^ducation curriculums. HFi order to do this; we are 
interviewing deans and depcirtment heads of occupational education 
concerning the recent establishment of different (Occupational education . 
programs.' In order to get at how you go about your processes here at 
your institution, where the key decisions are made, what some of the 
key activities are, .we would like to ask you questions that fall in^o 
these kinds of areas. These different areas are: 

1. ! Program Identification 

2. Program and Course Development 

3. Program' Evaluation 

We realize that not everybody does this in this same type of procedure. 
In general, every bpdy goes throu g h an activit y v/h ere th ey_hav.e_.to 
identify the kinds 6f prqgrams that the institution is going to be in--' 
volved in. Then they move into the development of that particular 
area. And after it is laid out and the decision is made, they stop to 
develop it.^ In^so doing, there are a number of considerations that 
need to be*" considered before the specific decision of program^ execution 
is made. ^ « 

If you are indicating to me that you did a manpower survey, etc., 
could you at times please tell me why you did it? In other words, at 
the time that you are teHing me t\^at you did a particular activity or 
function and you considered certain factors, it would be helpful to us 
in determining why you made a particular decision if this will not in- 
terrupt your telling me the story of how you developed that particular 
program. While^you are talking 3bout a specific area, I may have some 
specific questions for each of the areas. — - 



The data are confidential. 
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^KEY QUESTION APPROCH 

Program Identtfication , ' ^ 

!• How was. ^ identified as a potential area at 

. ' Col lege?, 

2. Who was involved in doing this? 

3. When wasHt done? v. 

•4. How was it done? . / 

Program-and Course-Development 

1. <What I mean by program and course development is who put 
the program and courses together, how were they -put to- 
gether, who planned the facilities, who laid out how the 
students were to be recruited, ^tc. 

2. What did you do to "look further into this area to deter;mine 
^ whether or not you should go ahead and plan program and 

courses, etc. ' ' / 

3. Who did you consult? / 
Who was involved in helping you go ahead and develop this 

\ program? * , ' 

5« When you. determined that you were going ahead to develop 
this .program, who was involved in the development/ of the 
courses? 

6. "^Who helped plan the facilities? » ' 

7. ^Who. helped recruit/students? . 

8. When were these done? ' 

9. Did you use advisory committees, curriculum guides, etc., in 
setting up the, courses? 

10. ' Wer^ the facilities here, or did you have - to plan and build 
n ew f ac Hrtres? ~ 



Program Evaluation 

1. \Vhat have you .done in terms of evaluation of this program, 
or what has or is being planned? 

2. Who will be involved? 

3. What kinds of factors are you going to consider? 
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IDENTIFICATION OF OCCUPATIONAL EDUCATION CURRICULUM DECISION-MAKING 



I 



Proqram Identification 
'2 3 4 (5 |6^7 8 9 10 



Program Development 



1 2!3 



4 5 



Program 
Evaluotion 



O Activ}tN>s Completed (doLpg). 
"n^^^t^. to Explore the Occupational Area 



m 

'Mi 



^ty/(^mpletedj^l Survey 
-ifS^. tboked at Old Manpower Survey Data 



l;'Determined No. of Target Population 



Determined Aspirations, Char. 8* Interest 



:7f*lVCompleted Job Analysis Survey 



I 



12 . Looked at, Programs in Other Institutions 
3: Held Meeting With: 



j;44H Re>>ort Sent for Approval 



i 



;:Dcveloped Specific Courses 



'/Recruited Staff 



^^7. B5 ^fu;ted Students 
"Mjt^ Planned Facilities 



I Detcrrrfncd What Equipment to Buy 



. Completed Survey of Drop outs^ 
TAsked Students to Evaluate Program 
. Ev<i)uation of Program 




Completed Follow-up Survey of Graduates 



Evaluated Staff 



. f mnloyer Evaluations 



Did evaluation feedback into program? 



I. Determined a Budget 



^9. Advisor y Committee/bub'Uommittce 
J. Interested Parents 



Fnculty 



'lj\^2. Com.nunity Organizations 



1 



industrial Relations & Other Related Committees 
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. Ip'torested Businessmen 



' ^b*^^ on an(< Managem ent Or( |anizations_ 
State Consultants , 



^ Stu dents Expre ssed an Interest 



). Curriculum Guidelines 



Manpower Data ^ 1 

71. Local Money Available 

State and Federal Money Available 



i'W. Physical Facilities Available 



Dnstraints (influences affecting the decision) 



v^^^. Money 




t^itJ. Pplilical (influential p eople) 



I Time 



^ ortise 
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